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ABSTRACT 

We are developing an approach for generating ground 
reference data in support of a project to produce a 30m 
impervious cover data set of the entire Earth for the years 
2000 and 2010 based on the Landsat Global Land Survey 
(GLS) data set. Since sufficient ground reference data for 
training and validation is not available from ground surveys, 
we are developing an interactive tool, called HSegLearn, to 
facilitate the photo-interpretation of 1 to 2 m spatial 
resolution imagery data, which we will use to generate the 
needed ground reference data at 30m. Through the 
submission of selected region objects and positive or 
negative examples of impervious surfaces, HSegLearn 
enables an analyst to automatically select groups of 
spectrally similar objects from a hierarchical set of image 
segmentations produced by the HSeg image segmentation 
program at an appropriate level of segmentation detail, and 
label these region objects as either impervious or non- 
impervious. 

Index Terms — Image Processing, Image 

Segmentation, Geography, Urban Areas. 

1. INTRODUCTION 

We are engaged in a project to produce a 30m impervious 
cover data set of the entire Earth for the years 2000 and 
2010 based on the Landsat Global Land Survey (GLS) data 
set. The GLS data from Landsat provide an unprecedented 
opportunity to map global urbanization at this resolution for 
the first time, with unprecedented detail and accuracy. 
Moreover, the spatial resolution of Landsat is absolutely 
essential to accurately resolve urban targets such as 
buildings, roads and parking lots. Linally, with GLS data 
available for the 1975, 1990, 2000, and 2005 time periods, 
and soon for the 2010 period, the land cover/use changes 
due to urbanization can now be quantified at this spatial 
scale as well. 

Our approach works across spatial scales using very 
high spatial resolution commercial satellite data to both 
produce and evaluate continental scale products at the 30m 
spatial resolution of Landsat data. We are developing 
continental scale training data at lm or so resolution and 
aggregating these to 30m for training a regression tree 


algorithm. Because the quality of the input training data are 
critical, we have developed an interactive software tool, 
called HSegLearn, to facilitate the photo -interpretation of 
high resolution imagery data, such as Quickbird or Ikonos 
data, into an impervious versus non-impervious map. 
Previous work has shown that photo -interpretation of high 
resolution data at 1 to 2m spatial resolution will generate an 
accurate 30m resolution ground reference when coarsened 
to that resolution. Since this process can be very time 
consuming when using standard clustering classification 
algorithms, we are looking at image segmentation as a 
potential avenue to not only improve the training process 
but also provide a semi-automated approach for generating 
the ground reference data. 

We are developing a photo-interpretive tool, 
HSegLearn, which takes as its input a hierarchical set of 
image segmentations produced by the HSeg image 
segmentation program [1, 2]. HSegLearn lets an analyst 
specify pixel locations as being either positive or negative 
examples, and displays a classification of the study area 
based on these examples. Lor our study, the positive 
examples are examples of impervious surfaces and negative 
examples are examples of non-impervious surfaces. 
HSegLearn searches the hierarchical segmentation from 
HSeg for the coarsest level of segmentation at which 
selected positive example locations do not conflict with 
negative example locations and labels the image 
accordingly. The negative example regions are always 
defined at the finest level of segmentation detail. The 
resulting classification map can be then further edited at a 
region object level using the previously developed 
HSegViewer tool [3]. 

After providing an overview of the HSeg image 
segmentation program, we provide a detailed description of 
the HSegLearn software tool. We then give examples of 
using HSegLearn to generate ground reference data. 

2. THE HSEG IMAGE SEGMENTATION PROGRAM 

A popular approach for performing image segmentation is 
best merge region growing. An early example is the 
Hierarchical StepWise Optimization (HSWO) approach of 
Beaulieu and Goldberg [4]. HSWO is an iterative form of 
region growing, in which the iterations consist of finding the 


most optimal or best segmentation with one region less than 
the current segmentation. 

In complex scenes, such as remotely sensed images of 
the Earth, objects with similar spectral signatures (e.g., 
lakes, agricultural fields, buildings, etc.) appear in spatially 
separated locations. In such cases, it is useful to aggregate 
these spectrally similar but spatially disjoint region objects 
together into groups of regions objects that we call region 
classes. This aggregation may be performed as a post- 
processing step. However, best merge region growing, as 
exemplified by HSWO, may be modified to integrate this 
aggregation directly into the region growing process. This is 
the basis of the Hierarchical Segmentation (HSeg) 
algorithm. 

HSeg produces a segmentation hierarchy a set of 
several image segmentations at different levels of detail in 
which the segmentations at coarser levels of detail can be 
produced from simple merges of regions at finer levels of 
detail. This hierarchy may be useful for applications that 
require different levels of image segmentation details 
depending on the characteristics of the particular image 
objects segmented. An important feature of a segmentation 
hierarchy that distinguishes it from most other multilevel 
representations is that the segment or region boundaries are 
maintained at the full image spatial resolution for all levels 
of the segmentation hierarchy. 

The key unique aspect of HSeg is its tight intertwining 
of region growing segmentation, which produces spatially 
connected region objects, with non-adjacent region object 
aggregation, which groups sets of region objects together 
into region classes. HSegLearn takes advantage of this 
region object classification to extend its positive and 
negative example labeling throughout the entire image 
based on just a few examples. 

3. THE HSEGLEARN PHOTO- 
INTERPRETATION TOOL 

HSegLearn provides a convenient graphical user interface 
for designating region classes from the HSeg segmentation 
hierarchy as positive and negative example region classes. 
One way an analyst can do this is by selecting a pixel 
location in a display panel showing an RGB rendition of the 
image data or in a display panel of the current image 
labeling, and having HSegLearn highlight (in yellow) the 
region class containing the selected pixel. The analyst can 
either then select another region class for highlighting, or 
submit the currently highlighted region class as a positive or 
negative example class. Another way an analyst can 
highlight region classes is by selecting the “Circle Region of 
Interest” option on one of the display panels and circling an 
area in the display panel. HSegLearn will then highlight 
every region class contained within the circled area. The 
analyst can either then select another region class or group 
of region classes for highlighting, or submit the currently 


highlighted region classes as positive or negative example 
classes. 

Region classes submitted as negative examples are 
displayed in red at the finest resolution of the segmentation 
hierarchy. However, region classes submitted as positive 
examples are displayed in green at the coarsest resolution of 
the segmentation hierarchy at which the region class spatial 
extent does not overlap any submitted negative example 
region class at the finest resolution of the segmentation 
hierarchy. For the case for which a region class is submitted 
as both a positive and negative example, this region class is 
designated as ambiguous and colored purple in the display. 
An ambiguous class designation can be overridden by a 
subsequent resubmission as either a positive or negative 
example class. 

4. GENERATING GROUND REFERENCE 
DATA WITH HSEGLEARN 

In this section we provide an example of the generation of 
ground reference data with the HSegLearn photo- 
interpretation tool. HSegLearn provides a graphical user 
interface (GUI) (Fig. 1) through which an analyst can bring 
up a display of an RBG rendition of the image data and a 
display of the current labeling of the scene (initially blank). 

Our test scene is a 5512x4648 4-band pan-sharpened 
Ikonos image from the Centralia, MO area at lm spatial 
resolution. A portion of that scene is displayed in Fig. 2(a). 
In our example, we first click on a pixel in the middle of a 
dark roof and the bottom of the image and highlight it by 
clicking on the button “Click Here to Highlight the Region 
Class Located at the Last Left Mouse Button Click.” The 
result after “selecting” this region class as a “positive 
example” is shown in Fig. 2(b). 

After we “submit” the selected region class as a 
“positive example,” HSegLearn searches the HSeg 
segmentation hierarchy for the coarsest segmentation that 
contains this dark roof that also does not conflict with any 
region classes submitted as negative examples. Since we 
haven’t submitted any negative examples yet, HSegLearn 
returns the coarsest possible segmentation from the 
segmentation hierarchy, as displayed in Fig. 3(a). 

We now need to provide HSegLearn with some 
negative examples. We can do this by circling a region of 
interest in, for example, an agricultural field, as shown in 
Fig. 3(b), and the highlighting and selecting the HSeg region 
classes (at the most detailed segmentation level in the 
segmentation hierarch) contained in the region of interest as 
negative examples. We can also highlight and select 
individual region classes as negative examples in the same 
way we highlighted and selected the dark roof positive 
example. The result after highlighting and selecting several 
HSeg region classes as negative examples is shown in Fig. 
4(a). 



Figure 1. An example of the HSegLearn graphical user 
interface. 



(a) (b) 

Figure 2. (a) RGB rendition of a portion of an Ikonos image, 
(b) The current region labeling after selecting the HSeg 
region class associate with a dark roof. 



(a) (b) 

Figure 3. (a) The current region labeling after submitting the 
HSeg region class associate with a brown roof as a positive 
example (with no negative examples yet provided), (b) The 
current region labeling after circling a region of interest in 
an agricultural field. 


We now submit our negative examples to HSegLearn. 
When we do this, HSegLearn adjusts the selection of the 
hierarchical segmentation level of the previously submitted 
positive example(s) so that the positive example labeling 




(a) (b) 

Figure 4. (a) The current region labeling after selecting (and 
not yet submitting) several negative examples, (b). The 
current region labeling after submitting several negative 
examples. 



(a) (b) 

Figure 5. (a) The current region labeling after several 
positive examples of impervious surfaces are selected (roofs 
and parking lots), (b). The current region labeling after the 
additional positive examples are submitted to HSegLearn. 


does not conflict with the submitted negative examples. The 
result is shown in Fig. 4(b). Note that most of the area 
previously highlighted as impervious (green) is now left 
unlabeled. 

To increase the labeling of positive examples of 
impervious surfaces we must provide HSegLearn with more 
positive examples. The result of selecting another roof and a 
parking lot as positive examples is displayed in Fig. 5(a). 
Note that the selected region classes are highlighted per the 
finest segmentation level in the HSeg segmentation 
hierarchy. After we submit these positive examples, 
HSegLearn, searches the HSeg segmentation hierarchy for 
the coarsest segmentation levels in the HSeg segmentation 
hierarchy for labeling the image, as shown in Fig. 5(b). The 
result is a very complete and accurate labeling of this 
portion of the image as impervious (green) versus non- 
impervious (red or black). 


5. DISCUSSION 


The HSegLearn tool solves a key problem we have 
encountered with other approaches we have attempted to use 
for generating ground reference data from high spatial 
resolution imagery. This key problem was how to define the 
most effective level of segmentation or clustering detail. 
When too few clusters or region segments were selected, the 
classes of interest could not be separated, but when too 
many clusters of region segments were selected, the labeling 
of the classes of interest became very inefficient. 
HSegLearn automatically selects the most effective level of 
segmentation detail for labeling the classes of interest (for a 
binary positive class/negative class labeling problem) based 
on a simple selection of positive and negative examples by 
an analyst. 

The effectiveness HSegLearn’ s automatic selection of 
level of segmentation detail is demonstrated in the example 
discussed in the previous section. With the selection of just 
a few positive example region classes, along with a 
judicious selection of a few negative example region 
classes, a very accurately labeling of the image into 
impervious versus non-impervious surfaces is quickly 
obtained. 
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